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I.  INTRODUCTION 


The  research  and  development  (R&D)  question  is:  "What  impact  does 
a  head-slaved  or  an  eye-slaved  visual  system  have  on  the  subjective 
perception  of  visual  lag  times?"  For  a  theoretical  evaluation  of  this 
question,  lag  time  is  defined  to  be  the  difference  between  the  time  in 
which  the  simulated  event  occurs  and  that  of  the  real  system  (Figure  1). 

REACTION  (REAL) 

T  REACTION 
-  (SIMULATED) 

t 


FIGURE  1:  DEFINITION  OF  LAG  TIME 


Lag  times  below  100  msec  in  flight  simulator  visual  systems  are  not 
subjectively  discernible.  However,  detailed  studies  in  this  area 
indicate  that  performance  deterioration  can  be  objectively  measured, 
even  when  time  delays  as  small  as  60  msec  exist  in  the  visual  feedback 
loop  (Queijo  &  Riley,  1975). 

The  definition  of  lag  time  makes  it  clear  that  the  lag  of  a 
simulator  can  be  subjectively  perceived  only  if  the  perceiver  is 
experienced  with  the  real  system.  Performing  an  action  like  steering  a 
car  or  piloting  an  aircraft  raises  the  conciousness  of  the  experienced 
driver  or  pilot  to  a  state  of  expectation  about  the  upcoming  reaction. 
The  driver  or  pilot  will  then  compare  this  expectation  value  with  the 
actual  event.  Thus,  the  more  experienced  pilots  will  be  most  sensitive 
to  a  time  delay  in  a  flight  simulator. 

In  addition,  lag  perception  will  depend  on  the  difference  between 
the  simulated  and  the  real  position  at  a  given  time.  This  difference 
(DELTA  P  in  Figure  ?)  increases  significantly  with  increasing 
acceleration  capability  of  the  system.  Angular  position  changes  and 
angular  accelerations  are  essential  for  high  fidelity  visual  system 
performance. 
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ACCELERATION  CAPABILITY 


Experience  of  the  ooserver  and  acceleration  of  the  system  are  the 
two  primary  parameters  that  need  to  be  examined  for  evaluating 
conventional , as  well  as  heau-slaved  and  eye-slaved,  visual  systems. 

In  conventional  visual  systems,  the  reaction  of  the  visual  scene 
depends  on  the  'vement  of  the  aircraft.  Pilots  experience  this 
movement  only  during  their  flight,  hours,  and  they  have  to  adapt  each 
time  they  change  planes.  Furthermore,  the  most  rapid  movement  of  a 
modern  fighter  aircraft  is  a  roll  with  typical  accelerations  of 
approximately  600  deg/sec^  and  maximum  values  up  to  1200  deg/sec9. 

In  head-slaved  and  eye-slaved  visual  systems,  the  reaction  of  the 
visual  scene  depends  on  the  movement  of  the  aircraft  superimposed  on 
that  of  the  head  and  eyes.  Head  and  eye  movements  are  experienced  and 
practiced  with  every  movement  in  life.  Head  rotations  can  be  performed 
with  accelerations  in  the  range  of  2000  deg/sec^,  and  peak  values  up 
to  6000  deg/'sec^  are  reported.  Maximum  eye  accelerations  may  be  as 
much  as  53,000  deg/sec^  (Lobb,  Barber  &  Murray,  1979). 
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Table  1  presents  a  convenient  comparison.  The  table  indicates  that 
lag  times  will  become  subjectively  more  apparent  in  head-slaved  and 
eye-slaved  visual  systems  than  in  conventional  visual  systems. 


Table  1.  Comparison  of  Display  Configurations 


conventional 

head-slaved  eye-slaved 

experience  in 

trained  skill ; 

lifetime  experience  and 

real  scene 

limited  flight  hrs; 

trai ni ng; 

dynamics 

changes  with  aircraft 

only  small  gradual  changes 

normal 

acceleration 

deg/sec^ 

600 

2,000  ? 

peak 

acceleration 

deg/sec^ 

1,200 

6,000  53,000 

Due  to  technical  limitations,  there  is  no  foreseeable  reduction  in 
lag  times.  Alternative  methods  must  be  developed  to  make  the  temporal 
characteristics  of  head-slaved  aui  eye-slaved  systems  acceptable.  One 
such  method  involves  predicting  the  head  and  eye  movement.  This  paper 
discusses  approaches  to  such  a  prediction  for  applications  in  a 
head-slaved  visual  system. 


II.  METHOD  TO  MEASURE  ROTATIONAL  ACCELERATION 

Before  developing  prediction  algorithms,  data  on  actual  head 
movements  were  gathered.  Such  records  are  valuable  in  characterizing 
head  dynamics  and  serve  as  test  data  for  candidate  algorithms. 

To  produce  a  cumplete  picture  of  a  movement,  the  head  position, 
velocity,  and  acceleration  information  must  De  available.  Although  the 
latter  two  can  be  derived  from  the  first  one  by  differentiation,  it 
seemed  to  be  impractical  because  instrumentation  noise  may  cause 
serious  restrictions  for  subsequent  differentiations.  For  that  reason, 
a  method  of  measuring  acceleration,  then  deriving  velocity  and  position 
by  integration,  was  selected. 

Since  linear  accelerometers  were  available,  two  were  attached  to  a 
helmet  to  measure  the  rotation  about  one  axis.  The  two  accelerometers 
were  oriented  in  equal  but  opposite  direction  and  the  angular 
acceleration,  AA,  was  derived  from  the  linear  acc o',  ation  values,  LAI 
and  LA2,  according  to  the  formula 
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3600 

AA  - - (LAI  +  LA2 

2  PI  D 


with  D  as  the  distance  between  the  two  accelerometers  (Figure  3).  Due 
to  the  geometric  arrangement,  the  averaging  of  the  linear  acceleration 
values  has  two  advantageous  effects:  first,  the  influence  of  gravity 
is  cancelled  out,  and  second,  the  center  of  rotation  does  not  need  to 
coincide  with  the  center  of  the  accelerometer  configuration.  Thus,  it 
was  required  only  to  adjust  two  linear  accelerometers  to  exactly 
antiparallel  positions  within  a  plane  perpendicular  to  the  axis  of 


LA  1  = 


lA  2  = 


211  0/2 

360° 

2 'IT  D/2 
360° 


AA  +  G  •  cosoC 


AA  -  G  •  cosoc 


AA  = 


360° 

2TTD 


(LAI  +  LA  2) 


FIGURE  3:  ACCELEROMETER  CONFIGURATION 
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The  overall  error  was  estimated  to  be  less  than  3%.  It  was 
composed  of  the  error  of  the  sum  signal  (LA1+LA2)  and  the  error  in 
measurement  of  the  distance  (0)  between  the  two  accelerometers. 

D  was  29  cm  and  could  be  measured  with  an  accuracy  of  1%.  The 
error  in  tne  sum  signal  was  generated  from  several  sources: 

1.  The  tilt  with  respect,  to  the  real  axis  of  rotation  was 
estimated  to  be  smaller  than  1U  degrees,  thus  contributing  an  error  of 
less  than  2%. 

2.  The  error  in  the  parallel  adjustment  of  the  two 
accelerometers  could  be  regarded  to  be  constant  during  one  period  of 
measurement;  it  was  accounted  for  digitally  in  subsequent  data 
processing. 

3.  Nonlinearity  and  zero  offset  of  the  accelerometers  as 
claimed  by  the  manufacturer  were  negligible. 


III.  PROCEDURE  OF  DATA  SAMPLING 

The  output  of  the  linear  accelerometer  configuration  was  applied  to 
an  analog-to-digital  board  of  a  microcomputer.  A  frequency  generator 
triggered  the  microcomputer  to  sample  the  acceleration  data  at  120  Hz. 
Each  sampling  period  covered  2  seconds. 

Start  and  end  of  a  sampling  period  was  indicated  by  a  bell.  In 
sync  with  the  start  and  end  bell,  one  out  of  e^ght  light  emitting 
diodes  (LED)  was  switched  on  and  off  by  the  microcomputer.  The  eight 
LEDs  were  arranged  in  an  array  to  stimulate  refixations  at  approxi¬ 
mately  +10,  +20,  +30,  and  +40  degrees.  E  i  session  consisted  of  24 
sampling  periods  where  the^lEDs  were  switched  in  a  pseudo-random 
sequence. 

Data  recording  was  performed  for  six  subjects  consisting  of  two 
pilots  and  four  task  scientists,  including  the  author,  from  the 
Operations  Training  Division.  The  subjects  had  been  asked  to  refixate 
as  fast  as  possible  but  to  keep  the  head  still  before  the  start  bell 
rang  and,  again,  as  soon  as  the  new  position  was  acquired  until  the  end 
bell  rang,  thus  performing  a  step  response. 

Yaw  movements  were  recorded  for  all  six  subjects.  Pitch  and  roll 
movements  were  recorded  for  three  of  the  subjects.  In  order  to  get 
some  indication  of  the  influence  of  the  helmet,  its  weight  was 
increased  from  4.75  pounds  to  12.75  pounds,  thereby  increasing  its 
moment  of  inertia  from  135  pounds  x  inch^  to  525  pounds  x  inch^. 

The  corresponding  movements  were  recorded  for  three  subjects. 
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IV.  DATA  ANALYSIS 


In  Tables  2  and  3,  the  maximum  values  for  acceleration  and  velocity 
are  presented.  The  remarkable  differences  between  the  subjects  can  be 
explained  by  different  eye/head  correlations  In  order  for  the  subject 
tc  fixate  on  a  new  target,  a  combined  head  and  eye  movement  was 
involved.  Some  subjects  tended  to  large  eye  excursions,  thus  achieving 
the  same  task  with  a  smaller  and  slower  head  movement. 


Table  2.  Maximum  Recorded  Acceleration  (deg/sec^) 


Subject 

YAW 

YAW+WEIGHT 

PITCH 

ROLL 

CH 

653 

CS 

2,219 

1,390 

968 

PC 

1,338 

TL 

968 

460 

810 

729 

TS 

1,432 

1,482 

UL 

1,539 

1,517 

1,447 

2,038 

Subject 

Table  3. 

YAW 

Maximum  Recorded 

YAW+WEIGHT 

Velocity 

PITCH 

(MAX)  (deg/sec) 

ROLL 

CH 

91 

CS 

187 

195 

90 

PC 

144 

TL 

88 

76 

99 

124 

TS 

136 

179 

UL 

154 

163 

124 

240 
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The  required  task  of  fixating  on  a  target  without  knowing  its 
location  in  advance  did  not  produce  the  reported  overall  maximum  values 
of  6000  deg/sec^  for  acceleration  and  370  deg/sec  for  velocity  (Lobb, 
Barber  A  Hurray,  1979),  After  changing  the  task  to  performing  a  rapid 
movement  in  a  predetermined  direction  (without  requiring  refixation  at 
a  specific  point),  a  maximum  acceleration  of  5,718  deg/sec*  and  a 
maximum  velocity  of  up  to  437  deg/sec  could  be  recorded. 

Figure  4  shows  representative  acceleration,  velocity,  and  position 
plots  for  a  single  movement.  The  curves  were  similar  for  all  subjects, 
as  well  as  for  the  three  axes  of  rotation.  The  steps  in  the  plots 
correspond  to  the  120  Hz  sampling  rate. 
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Any  arbitrary  force  can  be  treated  as  a  succession  of  impulses.  The 
inspection  of  head  acceleration  plots  revealed  resolvable  impulses,  not 
sharper  than  a  limited  magnitude.  This  limit  can  be  considered  to  be  a 
measure  of  the  latency  of  the  head  movement.  For  each  session,  between 
60  and  150  impulses  could  be  resolved.  After  normal izati on  (e.g., 
division  by  maximum  value),  they  were  superimposed  on  one  another,  thus 
creating  a  histogram  as  in  Figure  5.  (The  blank  area  in  the  middle 
indicates  the  existence  of  a  unit  impulse.)  To  get  a  clearer  picture 
of  the  unit  impulse,  a  weighted  average  of  all  impulses  was  generated. 
The  result  was  a  rather  smooth  pylse  shape  like  that  in  Figure  6.  It 
should  be  mentioned  that  no  curve-fitting  process  was  applied  to  gain 
this  pulse  presentation.  The  response  to  a  unit  impulse  is  the 
sharpest  step  increase  in  velocity  the  head  is  able  to  perform. 


1000  MSEC 


FIGURE  5:  PULSE  HISTOGRAM  FIGURE  6:  UNIT  IMPULSE 
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Table  4  lists  the  pulse  widths  at  half  maximum  of  the  unit  impulses 
of  each  session.  The  additional  weight  on  the  helmet  showed  the 
expected  effect  of  broadening  the  pulse  shape,  but  the  low  magnitude  of 
this  effect  was  '»omewhat  surprising.  The  data  In  Table  4  show  an 
increase  in  the  pulse  width  of  about  30  %,  whereas  the  helmet  weight 
had  been  increased  by  168  %,  thereby  increasing  the  moment  of  inertia 
by  290  %. 


Table  4.  .Unit  Impulse  Width  at  Half  Maximum  (msec) 


Subject 

YAW 

YAW+WEIGHT 

PITCH 

kOLL 

CH 

108 

CS 

79 

100 

58 

PC 

75 

TL 

67 

10b 

58 

67 

TS 

83 

83 

Ul 

96 

133 

83 

83 

In  spite  of  the  additional  stress  to  balance  more  than  twice  the 
helmet  '/eight,  the  neck  muscles  seemed  to  compensate  well  for  the 
additional  moment  of  inertia.  It  can  therefore  be  assumed  that  the 
normal  helme+  data  correspond  very  closely  to  bare  head  behavior. 
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V.  RESULTS  OF  NONLINEAR  PREDICTION 


Suppose  It  takes  two  frames  (one  frametime  =  33-1/3  msec)  to 
generate  a  visual  scene  and  another  frame  to  display  it  (update 
rate  =  30  Hz).  Without  any  prediction,  the  simulated  scene  would  lag 
between  two  and  three  frametimes,  depending  on  the  picture  element 
under  consideration.  Figure  7  shows  the  corresponding  simulated 
position  with  respect  to  the  real  position  from  Figure  4.  The  steps  in 
the  plot  are  due  to  the  30  Hz  update  rate. 

The  simplest  way  to  perform  a  prediction  is  to  extrapolate  the 
position  with  constant  velocity.  An  appropriate  choice  for  the 
extrapolation  time  would  be  two  and  one-half  frametimes  (=  83-1/3 
msec).  This  linear  prediction  may  be  sufficient  as  long  as  the 
movement  does  not  change  too  rapidly,  or  to  be  more  precise,  as  long  as 
the  actual  change  of  velocity  over  the  time  of  prediction  is  small 
enough.  Figure  7  shows  the  respective  linearly  predicted  position. 
Velocity  was  assumed  to  be  calculated  from  position  information  sampled 
at  60  Hz. 


WITHOUT  AND  WITH  LINEAR  PREDICTION 
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Since  the  real  velocity  changed  significantly  during  the  prediction 
time,  the  linear  predicted  position  still  deviates  from  the  actual 
position,  showing  lags  and  overshoots. 

To  Improve  the  prediction,  the  change  in  velocity  (i.e., 
acceleration)  has  to  be  taken  into  account.  Two  observations  from  the 
sampled  head  movement  data  supported  the  idea  that  acceleration 
information  could  improve  the  prediction: 

1.  As  Figure  4  illustrates,  acceleration  reaches  significant 
values  some  time  before  the  resultant  position  change  becomes  apparent. 

2,  Once  a  certain  amount  of  acceleration  has  been  attained, 
it  cannot  be  switched  off  instantaneously  but  must  at  least  decrease 
along  the  tail  edge  of  the  unit  impulse. 

The  minimum  amount  of  change  in  velocity  (DELTA  V  in  Figure  8) 
during  the  extrapolf  :ion  time  is  given  by  the  corresponding  integral 
over  the  momentary  mpulse.  Unfortunately,  the  calculation  of  this 
integral  requires  knowledge  about  the  exact  position  of  this  momentary 
impulse.  The  frequent  overlapping  of  subsequent  pulses  and  differences 
in  pulse  shapes  make  it  difficult  to  derive  this  information 
instantaneously. 


ACCELERATION 


EXTRAPOLATION  TIME 
FIGURE  8:  MINIMUM  CHANGE  OF  VELOCITY 
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So,  the  nonlinear  term  to  be  added  to  the  present  velocity  was 
approximated  by  the  product  of  the  present  acceleration  and  a  constant 
multiplier.  Integration  over  tne  Impulse  in  Figure  6  indicated  that 
this  multiplier  (MULT)  had  to  be  somewhere  between  0  and  100  msec. 
Several  trials  showed  that  an  optimum  value  existed  for  best  prediction 
results. 

To  account  for  overshooting,  which  occurred  under  certain 
conditions,  the  extrapolated  velocity  was  limited  to  a  maximum  value 
(MAX). 

Figure  9  shows  the  result  of  a  nonlinear  prediction  of  the  movement 
of  Figure  4  with  MULT  =  50  msec  and  MAX  =210  deg/sec.  To  recognize 
the  improvement,  these  plots  have  to  be  compared  with  those  in  Figure  7. 
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The  prediction  algorithm  written  in  BASIC  consisted  of  the 
following  statements: 


REM  DIFFERENTIATE  POSITION 
VANG  =  ( ANG-ANG0)/0EL 

REM  EXTRAPOLATE  VELOCITY  AND  ANGLE 
VEXT  =  VANG  +  ACC*MUL,T 
IF  ABS(VEXT)  >  MAX  THEN  VEXT=SGN( VEXT)*MAX 
AEXT  =  ANG  +  VEXT* EXT 

REM  MEMORIZE  OLD  POSITION 
ANGO  =  ANG 


Table  5  shows  the  average  values  for  MULT  for  each  subject  and 
axis  of  rotation.  There  is  no  exact  correlation  between  MULT  and  the 
pulse  widths  of  Table  4  since  MULT  is  a  function  of  the  whole  pulse 
shape  and  not  of  the  pul  e  width  alone. 


Table  5.  Multiplier  (MULT)  (msec) 


Subject 

YAW  YAW+ WEIGHT 

PITCH 

ROLL 

CH 

35 

CS 

35 

36 

34 

PC 

33 

TL 

36 

33 

26 

31 

TS 

38 

34 

UL 

41 

41 

34 

30 

Changes  in  the  values  given  in  Table  5  between  -10  msec  and  +20 
msec  did  not  affect  the  result  of  the  prediction  significantly. 
Increasing  MAX  was  not  critical  either,  but  it  must  not  be  less  than 
the  maximum  velocity  of  the  present  movement.  In  an  application  of 
this  prediction  scheme,  MAX  should  be  dynamically  adjustable  to 
account  for  individual  differences  and  changing  behavior  with  changing 
tasks. 
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The  prediction  algorithm  was  implemented  in  the  Fiber  Optic 
Helmet-Mounted  Display  (Figure  10)  using  rotational  accelerometers 
attached  to  the  helmet.  The  actual  application  showed  that  the 
position  values,  provided  by  the  mechanical  position  sensing  system, 
were  too  noisy  to  be  differentiated.  For  that  reason,  the  algorithm 
was  changed,  deriving  present  velocity  by  integration,  thereby  taking 
further  advantage  of  the  acceleration  data.  The  integrated  value  was 
reset  to  zero  each  time  the  differentiated  position  values  indicated 
zero  velocity,  in  order  to  minimize  the  integration  error. 

The  modified  prediction  proved  to  be  successful.  Smooth  scene 
movement  was  observed  with  significantly  decreased  lag. 


REM  DIFFERENTIATE  POSITION 
VANG  =  (ANG-ANGO)/DEL 

REM  RESET  INTEGRATION  CONSTANT 

IF  VANG*VANG0  <  0  THEN  VACC=0 

REM  INTEGRATE  ACCELERATION 

VACC  =  VACC  +  DEL*( ACC+ACC0)/2 

REM  EXTRAPOLATE  VELOCITY  AND  POSITION 
VFXT  =  VAfT  +  AlT*M!ll  T 
IF  ABS(VEXT)  >  MAX  THEN  VEXT=SGN(VEXT)*MAX 
AEXT  =  ANG  +  VEXT*EXT 

REM  MEMORIZE  OLD  VALUES 
ANGO  =  ANG 
ACCO  =  ACC 
VANGO  =  VANG 


input  parameters 

ANG 

ACC 

present  position 
present  acceleration 

output  parameters 

:  AEXT 

extrapolated  position 

time  constants 

:  DEL 

EXT 

time  between  present  and 
previous  position 
extrapolation  time 

velocities 

VANG 

VACC 

VEXT 

differentiated  position 
integrated  acceleration 
extrapolated 
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ACCELEROMETERS 


FIGURE  10:  FIBER  OPTIC 

HELMET-MOUNTED  DISPLAY 


VI 


SUMMARY 


In  head-slaved  or  eye-slaved  visual  systems,  lag  times  in  the 
visual  feedback  loop  are  more  apparent  than  they  are  in  conventional 
fixed  display  systems.  The  available  technology  of  digital  image 
generators  does  not  permit  lag  times  to  be  reduced  to  the  required 
amount.  Therefore,  appropriate  prediction  algorithms  have  to  be 
developed.  Accelerometers  were  used  to  measure  the  step  response  of 
the  head  in  three  axes  of  rotation.  It  could  be  shown  that  linear 
prediction  does  not  provide  the  necessary  accuracy  in  the  simulated 
position.  A  further  analysis  of  the  recorded  data  revealed  that  it  is 
possible  to  take  advantage  of  the  head's  latency  to  improve  the 
prediction.  A  simple  nonlinear  prediction  algorithm  based  on 
acceleration  data  was  successfully  implemented  in  the  Fiber  Optic 
Helmet-Mounted  Display. 
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